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Abstract 

We introduce the (private) entropy of a directed graph (in a new 
network coding sense) as well as a number of related concepts. We 
show that the entropy of a directed graph is identical to its guessing 
number and can be bounded from below with the number of vertices 
minus the size of the graph's shortest index code. We show that the 
Network Coding solvability of each specific multiple unicast network 
is completely determined by the entropy (as well as by the shortest 
index code) of the directed graph that occur by identifying each source 
node with each corresponding target node. 

Shannon's information inequalities can be used to calculate up- 
per bounds on a graph's entropy as well as calculating the size of the 
minimal index code. Recently, a number of new families of so-called 
non-shannon-type information inequalities have been discovered. It 
has been shown that there exist communication networks with a ca- 
pacity strictly ess than required for solvability, but where this fact 
cannot be derived using Shannon's classical information inequalities. 
Based on this result we show that there exist graphs with an entropy 
that cannot be calculated using only Shannon's classical information 
inequalities, and show that better estimate can be obtained by use of 
certain non-shannon-type information inequalities. 

Category E.4 Graph Theory, Information Theory, Network Coding and Cir- 
cuit Complexity 
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1 Introduction 



1.1 Main results 

Informally, one of the key problems in proving lower bounds on circuits is to 
identify and prove that certain Circuit topologies (e.g. small circuits) provide 
information bottlenecks for information flows. We introduce the notion of 
(private/public) graph entropy and show that this notion in a very precise 
way captures such information bottlenecks in the context of communication 
networks. More specifically we show that a given communication network 
with k source and k corresponding target nodes provide an information bot- 
tleneck (i.e. is unsolvable) if and only if the entropy (or public entropy) of 
is strictly less than k. In the seminal works 121] and [S] it was shown that 
Shannon's information inequalities (commonly known as Shannon's laws of 
information theory) are in general insufficient to identify specific information 
bottlenecks in communication flow problems. Relying on this result we will 
show that a similar type of result is valid in the context of graphs entropy. 
By combining Theorem 1, Theorem 9 as well as Theorem 10, and modifying 
the acyclic multiple unicast version of the Vamos graph introduced in [8] (by 
passing to the corresponding line graph and by identifying each source node 
with each corresponding target node) we show that the resulting graph has 
an entropy that cannot be calculated using Shannon's Classical information 
inequalities and that better bounds can be achieved by use of Zhang and 
Young's non-shannon-information inequalities. 

In the paper we introduce a number of new (directed) graph parameter^ 
These include the (private/public) entropy, the Shannon (private/public) 
entropy, the Zhang- Young (public/ private) entropy as well as the Doghtery- 
Freiling-Zeger (private/public) entropy. The Concepts are in general different 
from (but linked to) graph parameters that have already been extensively 
studied in the literature. In general we drop the preflx "private" (when no 
confusion is possible) and refer to the (private) graph parameters as Entropy, 
S-entropy, ZY-entropy and DFZ-entropy. 

^later and joint with Sun Yun, we found - partly by extensive computer searches - 
somewhat smaller and simpler examples 

^throughout the paper graphs are always assumed to be directed 
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1.2 Experimental results for small graphs 

Using computer calculations we tested millions of (small) graphs and found 
that quite different notions (different variants of guessing numbers and Graph 
Entropy) led to identical numbers on the vast majority of graph^ Key 
concepts (most of which we will introduce in this paper), like the Graph 
Entropy, the guessing number, the S-entropy and the ZY-entropy - gave 
identical results for the vast majority of graphs we tested. One interesting 
aspect is that although the different calculations usually lead to the same 
numbers for the graphs we tested by computer (using weeks of computer 
time), there appears to be a very sparse set of (small) graphs where the 
calculations leads to slightly different results. It should, however, be noticed 
that small graphs are atypical and it would be a mistake to expect that the 
computational findings for small graphs with < 10 nodes in general remains 
valid for large graphs. To illustrate the problem, we noticed for example 
that most small oriented graphs have entropy identical to the number of 
vertex minus the acyclic independence number (i.e. the number of vertices 
in the maximal induced acyclic subgraph). This is, we believe, not typical 
for example, for large complex oriented graphs (e.g. random tournaments). 

One curious fact (we discovered experimentally), is that the entropy for 
the vast majority of (small) graphs is an integer. More specifically, we in- 
vestigated (using computer calculations) many classes of graphs. The vast 
majority of "small" graphs (less than 10 nodes) seem to have integer or half 
integer Entropy (identical to their S-entropy and ZY-entropy) 

This is striking as there does not seem to be any obvious reason why 
graphs in general should have integer Entropy. If we chose for the complete 
graph Kn for each edge an edge direction (getting a tournament on n vertex) 
our computer calculations showed that there is 1 graph of type 2 non- 
isomorphic graphs of type K^, 4 non-isomorphic graphs of type i^4, 12 non- 
isomorphic graphs of type i^s, 56 non-isomorphic graphs of type Kq, 456 
non-isomorphic graphs of type K-j and 6880 of type and 191536 non- 
isomorphic graphs of type 

Our computer calculations showed that all, but one, of the 531 non- 
isomorphic graphs of type Kj for j < 7 have integer Entropy. We did not 

■^a more detailed account of these and many other experimental findings is being pre- 
pared in collaboration with Sun Yun 

■^we found a few graphs with Shannon Entropy and ZY-entropy being a third integer 
^we identified this sequence independently by use of the dictionary of integer sequences 
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investigate all graphs of type and |^ but random checks suggest that 
the majority of these graphs have integer Entropy and that most remaining 
graphs has half integer Entropy. We do not know if this pattern changes for 
graphs of type for n large. What we do know - we notice this in section 
7 - is that the entropy is an integer whenever the (private) entropy and the 
public guessing number (=number of vertices minus the smallest index code) 
is identical. 

Another striking point is that the (Shannon) Entropy for the vast ma- 
jority of (small) graphs we tested is identical to the (Shannon) Entropy of 
the dual graph where all edge directions have been reversed. To appreciate 
this we noticed that the actual calculations needed to workout these identical 
Entropies, are often very different, and do not seem to mirror each other in 
any obvious sense. 

1.3 Historical background 

In [23] based on [22] Valiant introduced the concept of a bi-parte graph 
representing n input output pairs, being realized by m common bits. It 
turns out that Valiant's concept of being realized by m common bits, is 
mathematically equivalent to the graph G naturally associated to the bi- 
parte graph having an index code of length m. The notion of an index code 
of a graph (as well as a minimal index code) was first introduced in [1] and 
further result on this graph parameter was presented in [3] and [15] To 
confuse matters further, the very same concept was independently referred to 
in [20] and [19] as G having information defect m. To avoid further confusion 
we will follow [H E] and [IS] and refer to the notion as the minimal index 
code. 

Given a graph G = (V, E) with vertex set {0, 1, 2, . . . , n — 1} we define 
for each )f:G{0,l,2,...,n — 1} the "shifted" graph G* = (V, -E*) with vertex 
set V and with and edge (fi,f2) G E exactly when {yi,V2 ®t)EE^ where 
f 2 © t is calculated modulo n. This construction was essentially presented in 

m- 

Valiant's shift problem asks whether for each e > for all sufficiently large 
values of n, each sparse graph G = (V, E) with vertex set {0, 1, 2, . . . , — 1} 
and at most n^"*"^ edges, can be shifted such that G* has minimal index code 

^each graph required a couple of hours computer analysis 
^we recall the definition in section 7 
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of length at least m. If it could be shown to be valid for m = 0(. — , ■,. ), 

>^ ^log(log(n))^' 

then a significant breakthrough in Boolean circuit complexity would follow 
|17j [T9] . In [22] Valiant conjectured that m < n/2 was not achievable, and 
before [19] it was not even known if m < n — o{n) could be achieved. However 
in [in] it was shown that in certain cases all cyclic shifts lead to graphs with 
a (surprisingly) short minimal index code of length less than m = {n — n^)/2. 
Despite this result, Valiant's original common information approach remains 
viable. The guessing number of a graph was introduced in [20] and was later 
related to Valiant's shift problem [19j . 

The notion of Guessing numbers [20] and the notion of (private) graph 
entropy has, in general, been guided by analysis of specific communication 
flow problems. Our notion of guessing number of a graph grew out of work on 
the matrix transposition problem that is another long-standing open problem 
in circuit complexity [24J . To illustrate some of the difficulties in solving the 
matrix transposition problem, in [18] it was shown that there exist network 
flow problems that can be only solved by use of non-linear boolean functions. 
More specifically we constructed a specific acyclic graph with 25 input nodes 
and 25 output nodes, such that any linear solution (over an alphabet of two 
letters) can only transmit 24 of the messages correctly, while there exists 
a non-linear solution that correctly transmits all 25 messages. In the same 
paper [IS] we presented a general construction, that allowed to lift this (and 
similar results) that are valid for alphabets with two letters, to alphabets with 
any number of letters. However, for each size of alphabet the construction 
leads to a new (and typically larger) graph. In ^ the authors showed that 
such an increase in graph size can be avoided and that there exists an acyclic 
communication network that has a non-linear solution over an alphabet of 
4-letters, but fails to have linear solution (even if the alphabet is organized 
as a vector space) over alphabets of any finite cardinality. The paper [7] was 
also significant as the authors used arguments from information theory to 
show unsolvability of specific communication networks. 

In [H] and [H] the authors showed that a network might be solvable over 
all alphabets, but not scalar-linearly solvable over any alphabet. 

In [15] the authors used a very interesting general construction (based on 
an earlier construction that Alon, used to answer a question about Shannon 
Capacity [I]). They showed that there exist graphs Gp^q with no linear index 
code of less that n^""^^^ bits, but with a non-linear index code of n"^^'' bits. 
It is still an open question if there exists solvable communication networks, 
that requires non-linear coding functions, and where the performance (e.g. 
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the network capacity) gets substantially reduced if all coding functions are 
linear. 

In [21] a we constructed a specific acyclic graph that we showed consti- 
tutes what we will call a one-way information bottleneck. More specifically in 
pi] a concrete acyclic graph with 38 source nodes and 38 target nodes were 
constructed, such that the messages fed into the source nodes in general can 
be sent to their corresponding target nodes, while the "dual" network where 
all edge directions have been reversed and the role of source nodes and target 
nodes have been reversed - is unsolvable (only 37 messages can be sent in 
that direction). Using the lifting method from |TH] for any size s < 2 of 
alphabet, it is possible to construct a one-way communication network that 
is solvable over alphabets of size s in one direction, but is unsolvable in the 
reverse direction. 

The one-way information bottleneck constructed in [21j was improved in 
[9] where it was shown that there is a network that has a solution in one 
direction, but is unsolvable in the opposite direction. In the unsolvable di- 
rection the network remains unsolvable even if the size of alphabet is allowed 
to have any finite size |^ It should be pointed out that this network in general 
(also for large alphabets) is unsolvable in both directions. It is an open ques- 
tion if there exists a network that is a genuine one-way bottleneck, that is 
solvable for any sufficiently large alphabet in one direction, but is unsolvable 
for any choice of alphabet in the opposite direction. It is also an open ques- 
tion if more dramatic one-way bottlenecks can be constructed (e.g. with a 
polynomial difference in performance). The graph in [15] is self-dual (in fact 
all edges are bidirectional), so this construction would need to be modified 
to work. In [12J and independently in [21] is was shown that networks that 
are solvable by linear coding functions never contain a one-way information 
bottleneck. 

2 The Guessing Number of a graph 

The Guessing number of a (directed) graph was first introduced in [20] and 
[in]. To explain this concept consider the following situation (based on [TS] ) 
where 100 players each has a die with s-sides (you may assume the die is a 
traditional die, i.e. that s = 6). Assume each person rolls their die. No-one 
is allowed to look at their own die, however each person has access to the dice 

^in the solvable direction only certain sizes of the alphabet is allowed 
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values of the remaining 99 people. One way of presenting this is that each 
player has the value of their own die stuck to their forehead. But, unlike the 
famous puzzle where the players can get additional information from other 
players' hesitation, in this game, all players have to "guess"- simultaneously 
and without any communication - what die value they have stuck to their 
forehead. What is the probability that all 100 players correctly "guess" the 
value of their own die? It turns out that this question is ill-posed since the 
probability depends on the general protocol adopted by the players. If, for 
example, each player assumes that the sum of all dice- values is divisible by s 
and each player from this assumption deduces the value of their own die - all 
players are correct when exactly the sum of the dice value is indeed divisible 
by s. Or equivalently if just one player is right (this happens with probability 
^) all players are right. Had the players guessed in a random uncoordinated 

fashion they would only all have been right with probability Thus 
adopting the "0-mod s" guessing strategy the players are s^^ times more 
likely to be right, than if they guess randomly in an uncoordinated fashion. 

This game can be modified and played on a directed graph G = (V, E). 
The idea is that each node represents a player, and again each player throws a 
die with s-sides. The value of each die is passed on along each edge. Thus the 
player at node j has access to only the dice values corresponding to players 
in nodes i where is an edge in the graph. The task is - as before - to 
find a guessing strategy that maximize the probability all n — \V\ players 
guess correctly the value of their die. 

The example with the 100 players corresponds to the complete graph Kiqq 
(with bi-directed edges) on 100 nodes. On i^ioo the players can do s^^ times 
better than pure random guessing, which is why we say that the complete 
graph Kioo has guessing number 99. In general: 

Definition : For each s G {2,3,4,...}, we define the guessing number 
g{G, s) of a graph G to be the uniquely determined a such that " is 
the probability all n players guess correctly their own die value ( assum- 
ing the players have agreed in advance an optimal guessing strategy). 
The general guessing number g{G) is defined as svLPg^2,z,...9{^-i 

More generally consider a pair (/, G) where / = (/i, /2, . . . , /„) is a fixed 
function with / : A" ^ A" and G — (V, E) is a directed graph (not nec- 
essarily acyclic, and possibly with self loops) with V = {1,2, .. . ,n}. The 
values xi,X2, ■ ■ ■ ,Xn e A are selected randomly and the task for player j is 
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to guess the value of fj{xi, X2, ■ ■ ■ , The task for the players is to find a 
general (deterministic) strategy that maximizes the probability that all play- 
ers simultaneously guess correctly. The guessing number g{G, f, A) of (/, G) 
is the uniquely determined a such that " is the probability each player j 
guess correctly the value fj{xi, X2, ■ ■ ■ , Xn) (assuming the players have agreed 
in advance an optimal guessing strategy). In this paper we consider in gen- 
eral only the case where / is the identity map, and we write g{G, s) instead 
of g{G, id, s). 



3 The Entropy of a graph 

One way of thinking about the entropy of a graph informally, is to view the 
graph G as representing a composite physical system where the nodes 
represent identical physical subsystems that can each be in some state s ^ A. 
The (directed) edges in the graph indicate possible causal influences. The 
(dynamic) behavior of the composed physical system is determined by speciflc 
functions assigned to the vertices representing the underlying physical laws. 
Like in physical systems subject to the laws of thermodynamics, the behavior 
of the functions is such that the overall entropy Hq of the system J^q is 
maximized. Informally, the Entropy of G is deflned as the maximal entropy 
Ho possible for the system JFq when the system is subject to the causal 
constraints indicated by G (the system is, as is assumed in thermodynamics, 
always in equilibrium i.e. all constraints are satisfied at any given moment). 

More formally, let G = {V, E) be a (directed) graph with the set V of 
nodes (n := |V^|) and set E CV xV of edges. Assume that each node j E V 
in G has assigned a stochastic variable xj selected from some flnite alphabet 
(or state space) A with s > 2 elements. For each probability distribution p 
on tuples {xi,X2, ■ ■ ■ ,x„) e A" we define an entropy function Hp such that 
for each subset 5" C {1, 2, . . . , n} the real number Hp{S) is given by: 

Hp{S):^ E p{S,v)\og,{-^) (1) 

where p{S,v) for v = {vi,V2, ■ ■ ■ ,Vn) G A"' is the probability that a tuple 
{xi,X2, . . . ,Xn) e A" is selected with Xs-, = Vs-^Xs^ = Vs2,---,Xs^ = 
where S — {si, S2, ■ ■ ■ , Su}- Let H denote any such entropy function Hp. 
For each vertex j in G we introduce the information equation: 
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H{j\ii,i2,-- ■,id) =0 



(2) 



where (ii, j), (z2, j), . . . , j) G -E is the edges with head j. We refer to 
these n information equations as the information constraints determined by 
G. Notice that the use of logarithm in base s in the definition ensures that 
the Entropy function H is normahzed. Thus it satisfies H{j) < 1 for each 



Informally, the equation states that there is no uncertainty of the value 
of the variable xj corresponding to vertex j if we are given the values of all 
the stochastic variables associated with the predecessor vertices of j. 

Definition : 

The (private) entropy E{G, s) of a graph G over an alphabet A of size 
s e {2, 3, 4, . . .} is the supremum of Hp{l, 2, . . . ,n) of all entropy func- 
tions Hp over A that satisfies the n information constraints determined 
by G. The general entropy E{G) (or just entropy) of a graph G is the 
supremum of the entropy E{G, s) for s = 2, 3, 4, . . .. 

As usual the conditional entropy is defined as H{X\Y) := H{X,Y) — 
H{Y) and the mutual information between X and Y is defined as I{X; Y) := 



H{X) + H{Y)~H{X, Y) where H{X, Y) is shorthand for H{XUY). In gen- 



eral we drop set clauses when possible and for example write H{ii,i2, ...,ir) 
instead of 22, v})- We have -^(0) = 0. 

An entropy function H has H{X) = if and only if X is uniquely de- 
termined, and has H{X\Y) = if and only X is a function Y (see 
for such and other basic properties of entropy functions). Shannon showed 
that entropy functions satisfy a number of information inequalities e.g. < 



H{X,Y) H{X,Y) < H{X) + H{Y), H{X,Y\Z) < H{X\Z) + H{Y\Z) 
and H{X\Y,Z) < H{X\Y) < H{X,Z\Y). These inequalities can all be 



obtained as a special case of Shannon's famous information inequality 



We will show: 

Theorem 1 : For each directed graph G and for each s G {2, 3,4,...,} the 

guessing number equals the entropy (i.e. E{G,s) = g{G,s)). 

Furthermore, the general entropy of a graph G is identical to the general 
guessing number of G (i.e. E{G) = g{G)). 



j = 1,2,. . . ,n. 



H{X, Y, Z) + H{Z) < H{X, Z) + H{Y, Z) 



(3) 
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This result allows us to calculate the Entropy of a graph using quite differ- 
ent methods. One can calculate lower bounds on the entropy, by constructing 
an explicit entropy function, but alternatively lower bounds can be obtained 
by providing a guessing strategy. For all (small) graphs we tested, and we 
believe most graphs in general, good upper bounds on the entropy can be 
calculated using Shannons Information inequalities or computationally more 
efficiently by use of the polymatoidal axioms we will introduce in the next 
section. In general - for the vast majority of (small) graphs we analyzed by 
computer- it is possible to find matching lower and upper bounds on E{G, s) 
in this fashion. 

The main reason we do not merge the Guessing number and the (pri- 
vate) Entropy into one concept is that we, as already mentioned, wants to 
introduce a number of concepts related to the Entropy of a graph (e.g.the 
Shannon-entropy, the ZY-Entropy and the DFZ-entropy of a graph) and al- 
ready in [211 US] have defined a number of distinct types of guessing numbers 
of a graph (e.g. the linear guessing number and the scalar linear guessing 
number). Joint with Danchev we also considered other types of guessing 
numbers defined using analogues to various well known routing protocols 
(e.g. fractional routing [5]). 

4 The S/ZY/DFZ graph entropy 

The entropy function H has a number of basic properties that are very useful 
in computing the entropy of a graph. In general there exist functions that 
satisfy all of Shannon information inequalities, but are not genuine entropy 
functions. An entropy-like function / on an n-element set V = {1, 2, . . . , n} 
is a map / from the subsets of V into R. We require that an entropy-like 
function has /(0) = 0. For subsets X, F C \/ we let /(X,r) := f\X U Y) 
and we define f{X\Y) as f{X\Y) := f{X,Y) - /(F). 

We say an entropy-like function / is a shannon entropy function if for 
X,Y,ZCVwe have /(Z) + /(X, Y, Z) < f{X, Z) + /(F, Z) 

As pointed out in [8] this information inequality (combined with -ff (0) = 
0), is known to be equivalent to the so-called polymatroidal axioms (for a 
map / : 2^ R): 

iS) fm = 

^as usual we use R to denote the real numbers 
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(ii) forX,FCr, /(X)</(F). 

(iii) /(X n F) + /(X U F) < /(X) + fiY). 

For an entropy-like function / : P{V) ^ R we define the mutual infor- 
mation If such that 

If {A; B\C) := f{A, C) + C) - f{A, B, C) - f{C) (4) 

The special case where C = 0, gives lf{A; B) := f{A) + f{B) - f{A, B). 

We say an entropy-like function is a Zhang-Young entropy function (or 
just ZY-entropy function) if for A, B,C, D (1 V we have 

2If{C; D) < If{A; B) + If{A- C U D) + SIfiC; D\A) + If{C; D\B) (5) 

It can be shown that each ZY-entropy function is a Shannon entropy function. 
Expressed in our terminology, Zhang and Young discovered [28] that while 
each entropy function is a ZY-entropy function, there exist Shannon entropy 
functions that fail to be ZY-entropy functions. 

The condition required for functions to be a ZY-entropy function can be 
written as 

U <W (6) 

where 

U ■.= 2f{C) + 2f{D) + f{A) + f{AV^B)+Af{Ai^Ci^D) + f{Bi^Ci^D) (7) 
and 

W := ^f{C UD) + 3/(A UC) + 3/(A U D) + f{B U C) + f{B U D) (8) 

Equality U = W happens in a number of cases e.g. when A = B = C = D, 
when A, B, C and D are independent sets with respect to / (i.e. /(X UF) = 
/(X) + f{Y) as well as when X, F G {A, B, C, D} are distinct. 

In [H] a number of new shannon information inequalities were presented 
and showed to be independent. We refer to such information inequalities 
(that can not be derived from the ZY-information inequalities) as DFZ- 
information inequalities. 

Somewhat informally (depending on which of these non-shannon infor- 
mation inequalities we consider) we say that a function / is a DFZ-entropy- 
like function if If satisfies the ZY-information inequality as well the DFZ- 
information inequalities. 



11 



Definition : 

Let G be a graph. We define the S-entropy /ZY-entropy /DFZ-entropy of 

G as the maximal value o//({l, 2, . . . , n}) for any S/ZY/DFZ entropy- 
like function f that satisfies the entropy constraints determined by G. 

For a graph G we let Es{G), EzviG) and Edfz{G) denote the S- 
entropy resp. ZY-entrop and DFZ-entropy of G. 

Proposition 2 : For each graph G and each s e {2, 3, 4, . . . , }, g{G, s) — 
E{G,s) < Edfz < EzviG) < EsiG). Furthermore the S-Entropy 
Es{G) is bounded from above with the number \V\ of vertices minus 
the acyclic independence number of G. 

Proof: The equality g{G, s) = E{G, s) follows from Theorem 1. The inequal- 
ity E{G,s) < E^FziG) follows from the fact that each entropy function is 
an DFZ-entropy-like function. Ejjpz{G) < Ezy{G) follows from the fact 
that part of the requirement of being a DFZ-entropy-like function is that 
it is ZY-entropy-like. The ZY-information inequalities are known to imply 
Shannon's information inequalities, which ensures that Ezy{G) < Es{G). 
For each S-entropy-likc function / that satisfies the information constraints 
for G, for any subset B of vertices for each vertex j with all tails in B, 
f{B) = f{j,B). Let A <Z V is any subset with the induced graph on 
V \ C being acyclic. Without loss of generahty, we can assume that the 
elements in V \ C are the elements {1, 2, . . . , r} and that there is no edge 
from any vertex i to a vertex j if i,j G {1, 2, . . . , r} and i > j. But, then 
fiG) = fiG, 1) = /(C, 1,2) = ..., f{V) and thus fiV) = f{G) < \G\. The 
number \G\ is at most \V\ minus the acyclic independence number of G. X 

5 The Entropy of the pentagon 

It is not hard to show that has entropy as well as S-entropy n — 1. The 
entropy (S-entropy) of an acyclic graph is (recall that the entropy H[j) 
of a node with in-dcgrcc vanish since H{j) = i/(j|0) = 0). In general 
the graph Gn oriented as a loop has entropy (S-entropy) 1. To illustrate the 
concepts with a less trivial example consider the pentagon C5 where all edges 
are bi-directed. 

The Shannon Capacity - a famous graph parameter that is notoriously 
difficult to calculate - of C5 was eventually shown to be y/5 — 2.23... We will 



12 



show that pentagon C5 have Entropy (as well as S-entropy, ZY-entropy and 
DFZ-entropy) E{C^) = 2.5 It turns out that the concrete Entropy over 
an alphabet A is 2.5 if and only if s = 1^41 G {2, 3, 4, . . .} is a square number. 
The entropy of C5 over an alphabet of two elements is log2(5) = 2.32.... 

First we show that C5 has S-entropy < 2.5 i.e that any S-entropy-like 
function / that satisfies the information constraints determined by C5 has 
/(1,2, 3,4,5) < 2.5. 

Assume that / satisfies Shannon Information inequality f{A, B, C) + 
/(C) < /(AC) + /(5,C). If we let A = = {3},C = {4,5} we 

get /(I, 2, 3, 4, 5) + /(4, 5) = /(I, 3, 4, 5) + /(4, 5) < /(1, 4, 5) + /(3, 4, 5) = 
/(1, 4) + /(3, 5) < /(I) + /(3) + /(4) + /(5). Thus /(I, 2, 3, 4, 5) < /(I) + 
/(3) + /(4) + /(5)-/(4,5). 

Next notice that /(1, 2, 3, 4, 5) -/(2,5) < /(3,4|2,5) = /(4|2,5) < 
/(4|5) = /(4,5)-/(5). 

Thus /(I, 2, 3, 4, 5) < /(2,5) + /(4,5)-/(5) < /(4,5) + /(2). Adding the 
two inequalities we get: 2/(1,2,3,4,5) < /(I) + /(2) + /(3) + /(4) + /(5) <5 
which show that the S-entropy (and thus the entropy, as well as the ZY- 
entropy and DFZ-entropy ) of C5 is at most 2.5. 

Next we show that the entropy of C„ is at least 2.5 if s the size s of the 
underlying alphabet is a square number i.e s = for some t G {2, 3,4, . . .}. 
According to Theorem 1 (the easy direction that g{G, s) < E{G, s)) it suffices 
to show that C5 has guessing number > 2.5. Assume that in each of the five 
nodes of C5 is a player with two dice each with t-sides. Each of the players has 
access to the dice values of their two neighbors. Assume that each player's 
dice are labeled L and R and they use a guessing strategy where each player 
assumes that their L-die has the same value as the left-hand neighbors R- 
die, and that their i?-die has the same value as the right-hand neighbors 
L-die. Notice that this protocol divides the 10 dice into 5 pairs with one 
L and one R die in each pair. Notice that the players are right exactly if 
each of the 5 pairs of matched L and -R-pairs have identical dice-values. This 

1 5 1 2 5 1 5—2 5 

happens with probability t ~ s ~ t • This shows that the Entropy 
(and thus the S-entropy, the ZY-entropy and DFZ-entropy) of the pentagon 
(with bi-directed edges) is at least 2.5. 

To show that the entropy can only be achieved when s = \A\ is a square 
number, we use the non-trivial direction of Theorem 1 {E{G,s) < g{G,s)) 

Joint with Dantchev we showed that Ck in general have (Shannon) Entropy | for any 
> 4 
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and show that there is no guessing strategy that achieve guessing number 2.5 
unless s is a square number. Assume the players fix a (deterministic) guessing 
strategy. There are different assignments of values {xi,X2, ■ ■ ■ ,Xn) G A"' 
so if we let m denote the number of assignments where all players guess 
correctly, the guessing number of 2.5 is achieved exactly when m — s'^^^. 
Since m is an integer the identity is only possible when s is a square number. 

Finally assume that the alphabet has size 2. In general the guessing 
number of a graph G = {V. E) over an alphabet of size s is of the form log^(m) 
for some m G {0, 1, 2, . . . , s'^'}. Thus, since the guessing number of log2(6) > 
2.5 is impossible to obtain, it suffices to show that there is a guessing strategy 
where all players can correctly guess their own assigned coin (coin rather than 
die since s = 2) value with probability ^. One such strategy is that each 
player assumes that the assigned coin values do not have three consecutive 
O's and not two consecutive I's. This condition is satisfied globally exactly 
when one of the 5 configurations occurs: 00101,10010,01001,10100,01010. 
Notice that whenever this condition is satisfied "globally", each player can 
(based on "local" information) deduce the value of their own coin value. J(t 

6 Proof of Theorem 1 

A guessing strategy Q induces a map "i/g that maps A"- to A". More specif- 
ically, maps each dice assignment to the guess made by the individual 
nodes, and thus maps each tuple {xi,X2, ■ ■ ■ , Xn) G ^l" to a tuple 
{zi,Z2, ■ ■ ■ , Zn) G A" representing the players' guess. 

All players guess correctly their own die value exactly when the assign- 
ment {xi,X2, . . . , Xn) of dice values is a fix-point for "ifg. From this we notice: 
Observation: For each guessing strategy Q let C = C{Q) C A" denote 
the set of fixpoints for the map : A" — > A'*. The guessing strategy Q 
corresponds to guessing number \ogg{m), where m is the number of fixpoints 
(i.e. m = \C\). 

The set C C A° of fixpoints of form a code with m codewords from 
A^. Let p = p{Q) be the probability distribution on A^ where each code 
word in C has the same probability (in fact probability ^) and each word in 

\ C has probability 0. Since each probability distribution p on A^ defines 
an entropy function Hp, this especially applies to the probability distribution 
p{Q), that therefore defines an entropy function Hpi^gy 

claim: If Q is a guessing strategy for the graph G, the corresponding entropy 
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function Hp(^g^ satisfies the information constraints determined by G. 

To see this consider a given vertex v with predecessor vertices wi, W2, ...,Wd- 
The guessing strategy Q has the guess z„ determined as a function of 
Xwi, Xyj^, . . . , Xw^- Thus each word (ci, C2, c„) e C that corresponds to a 
word where each player guess correctly their own assigned value, have c„ being 
a function of c^^ , , • • • , c^^ • Thus the entropy of given , Xyj^ , • • • , a;^^ 
is 0. In other words 

H{y\wi, W2, ■ ■ ■ , Wd) = 0. This shows that g{G, s) < E{G, s). 

The entropy functions that arise from guessing strategies are constant on 
their support Most entropy functions are non-constant on their support 
and do not correspond to guessing strategies. Potentially such entropy func- 
tions might turn the inequality g(G, s) < E{G, s) into a strict inequality. To 
prove Theorem 1 we need to show that this cannot happen i.e. we need to 
show that E{G,s) < g{G,s). Assume that Hp is an entropy function that 
corresponds to a general probability distribution p on A". The support of 
the entropy function Hp that corresponds to the probability distribution p, 
forms a code C = C(p) C A'^. As before we notice that the code C can be 
used to define an entropy function He that is constant on the C C A° and 
vanishes outside C. 

If Hp{X\Y) = then X is uniquely determined by Y. If g is a probability 
distribution on A" with the same support as p, then Hq{X\Y) = 0. Or in 
general if a collection of information constrains each is of the form Hp{X\Y) = 
for some probability distribution p, then the very same constraints are 
satisfied for all probability distributions q with the same same support as p. 
Thus since Hp and He corresponds to probability distributions with the same 
support (= C) the Entropy function Hp satisfies the information constraints 
of the graph G if and only if Hq satisfies the information constraints of the 
graph G. 

Furthermore, a basic result in information theory states that the sym- 
metric probability distribution on the set of support, maximizes the entropy. 
Thus Hp{l,2,...,n) < i7c(l,2,...,n). 

To complete the proof it suffices to observe that if He is an entropy 
function that is defined from a code C over an alphabet A, and satisfies the 
information constraints of the graph G, then the code C can arises from a 
guessing strategy Q over A. 

This observation can be obtained as follows: The code C satisfies the con- 

^^thc support of p is defined to be the set of elements in A" that has non-zero probabihty 
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straints that the letter cj is uniquely determined by the letters , , . . . , Cj^ 
where ii, 12, . . . ,id are the nodes with head j. For each player j this determine 
a (partial) guessing function. To make to function total, choose any value 
for the function (it does not matter which) on each tuple (xj^jXjj, . . . ,Xi^) 
that does not have = Cj^ , = Ci^, . . . , = Ci^ for some code word 
(ci, C2, . . . , c„) G C. If the players follow this guessing strategy, they are all 
correct for all assignments Xi,X2, ■ ■ ■ ,Xn) G C C ^\ (and possibly even for 
more words). This shows that E{G, s) < g{G, s). 4> 



7 Minimal index codes 

Assume that we play the guessing game on a graph G = {V, E) and with 
alphabet A. Assume that we have available a public information channel 
that can broadcast m messages and for each assignment of dice values we 
broadcast a single message to all the players. What is smallest value of m 
for which there exists a general protocol such that each player is always able 
to deduce his/her own assigned value? This concept, defined in terms of 
guessing games was introduced in [20| without knowledge of As already 
explained we will use the terminology of |1] and define the shortest index 
code i{G,s) of G (wrt. to A) as logg{m) where s = \A\ denotes the size of 
the alphabet A. 

Assume that a graph G has a minimal index code (over an alphabet with s 
letters) of length log^(m). Then by definition it is possible to broadcast one of 
m-messages and ensure that all players with probability 1 work out their dice 
values. This is (essentially) [^a factor better than the probability 

that the players could have achieved if they disregarded the information 
given from G and only had available a public channel broadcasting one of 
m messages. Using the analog to the definition of the guessing number, we 
define the public guessing number as n — logg{m) i.e. as n — i{G, s). 

It is possible to define the entropy of the length of the minimal index 
code of a graph G. Given a strategy S that achieve an index code with m 
messages (i.e. it possible to broadcast one in m public messages, in such a 
fashion that each node in G to deduce their own assigned value). The length 
of the code is log^(m). We define the entropy of the code as the entropy of the 
set of public messages with the probability distribution by which they will be 

^^essentially, since if m does not divide s" each of the m messages are not broadcast 
with identical probabilities 
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broadcasted (this is not always the symmetric probabihty distribution). We 
let ientro{G, s) dcnotc the minimal entropy (calculated using the logarithm in 
base s) of any index code for G. A priori there is no reason to believe that 
the minimal code contains the same number of code words as the code with 
the minimal entropy (the code might contain more words but if the occur 
with less uniform probabilities their entropy might be lower). 

The value of the entropy of the index code with minimal entropy can 
be calculated in a fashion that can be approximated by entropy-like func- 
tions (S-entropy-like, ZY-entropy-like as well as DFZ-entropy-like). As in 
the definition of the graph entropy we assume that each node j & V in G 
has assigned a stochastic variable Xj selected from some finite alphabet (or 
state space) A with s >2 elements. Let be a finite set of potential public 
messages. To ensure W is large enough we may assume W = A^. It turns 
out the size of - as long as it is large enough - does not affect the result. 
For each probability vector p on x W we define an entropy function Hp 
analogous to equation (1). More specifically, for each probability distribution 
p on tuples {xi,X2, . . . ,Xn,Xw) € A^ x W we define an entropy function Hp 
such that for each subset S" C {0, 1, 2, . . . , n, w} 
given by (1) i.e. by 

Hp{S):= piS,v)\og,i-^) (9) 

but where p{S,v) for v = {vi,V2, ■ ■ ■ ,Vn,Vw) E A"' x W is the probability 
that a tuple {xi,X2,. ■ ■ ,Xn,Xw) E A^ x W is selected with Xs-^ = Vsj^,Xs2 = 
Vs2,-- ■ ,Xs^ = Vs^ if S = {si, S2, . . . , Su} C {1,2, . . . ,n,w}. Let H denote any 
such entropy function Hp. 

For each vertex j in G we introduce the information equation: 

H{j\ii,i2, ...,id,w)=0 (10) 

where (ii, j), (^2, j), • • • , {id,j) E E is the edges with head j. We also add the 
information equation 

H{w\l,2,...,n) = (11) 

as well as the equation 

^•^where w is a new symbol distinct from 1, 2, . . . ,n 



the real number Hp{S) is 
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H{l,2,...,n) =n 



(12) 



The task is to find the entropy function H that satisfies all constraints and 
produce the minimal value of H{w) (over A). We refer to ientro{G, s) as 
the minimal entropy of an index code over an alphabet A = {1,2, . . . ,s} 
associated to G. 

Proposition 3 : The minimal value of H{w) (over A with s — \A\) is at 
mosti{G,s) i.e. ientro{G,s) <i{G,s). 

Proof: First we show that there exists an entropy function Hp that satisfies 
the constraints determined by G such that Hp{w) < i{G,s). Consider a 
strategy S where each player choose a function (that depend on all incoming 
values as well as the public message). Consider the set C C ^\ x W of all 
tuples {xi,X2, ...,Xn,w) E A^ X W where w is the message that according to 
the strategy S have to be broadcast when the players have been assigned the 
values xi,X2, ■ ■ ■ ,Xn e A^. The set C contains exactly elemnets. Let p 
be the symmetric probability distribution on C (probability of each element 
being ^) that vanish on A"- xW \ C. Let Hp : P{V) R he the entropy 
function corresponding to that probability distribution. Let m denote the 
number of messages w E W that can occur this way. The entropy Hp{W) 
is at most \ogg{'m) since this is the highest entropy possible on a set with m 
elements. Since i{G,s) = \ogg{m) this shows that H{w) < i{G,s). X 
Like in the definition of private entropy we consider entropy-like func- 
tions /. We define the S-minimal index code is{G) as the minimal value of 
f{w) that is possible to derive using Shannon's Information Inequalities (or 
equivalently the polymatroidal axioms). We define the ZY-minimal index 
code izviG) as the minimal value of f{w) that is possible to derive using 
the ZY-information inequalities. The DFZ-minimal index code ioFziG) is 
defined analogous by considering the minimal value of f{w) that is possible 
for a DFZ-entropy like function. 

Proposition 4 : i{G,s) > ientro{G,s) > ioFziG) > izY{G) > is{G). 

Proof: The inequality i{G,s) > iintro{G,s) follows from Proposition 3. 
Each entropy function is an DFZ-entropy-like function which shows that 

Hntro{G, s) > iuFziG). Part of the requirement of being a DFZ-entropy-like 
function is that it is ZY-entropy-like and thusioFziG) > izviG). Finally, the 
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ZY-condition is known to imply Shannon's information inequalities, which 
ensures that izviG) > ^siG). Jl» 
Using the analog to the definition of the private entropy, (and Propo- 
sition 3) we define the pubhc Entropy EP''^^'^{G,s) oi G = {V,E) over an 
alphabet with s letters as n — ientro{G, s) and the general public entropy as 
sup5^2 3,...E^^^^''{G, s). The public S-entropy is defined as n — is{G), the pub- 
lic ZY-entropy as n — izviG) and the public DFZ-cntropy as n — iDFz{G). 
With these definitions. Proposition 4 can be restated: 

Proposition 5 : g^^^^^G, s) < E^^^^^G, s) < E^^f{G) < E^f'^G) < 
Ef^^'\G) 

Example: The index code of C5 with minimal entropy has entropy 2.5 (i.e. 

C5 has pubhc entropy 5 — 2.5 = 2.5) over alphabets of size 4, 9, 16, 25, 

The length i{G^,s) of the minimal index code is also 2.5 over alphabets of 
size 4,9,16,25,.... 

To see this we first show that is{C^) > 2.5 (and thus that the public 
S'-entropy is < 2.5). Assume that / satisfies Shannon Information inequality 
/(A 5, C) + f{C) < f{A, C) + /(E, C). If we let A = {1}, B = {3}, C = 
{4,5} we get /(I, 2, 3, 4, 5, «;) + /(4, 5, «;) = /(I, 3, 4, 5, «;)+ /(4,5,«;) < 
/(1, 4, 5, w)+ /(3, 4, 5, w) = /(1, 4, w)+f{3, 5, w) < /(l)+/(3)+/(4)+/(5) + 
2f{w). Thus /(I, 2, 3, 4, 5, to) < /(I) + /(3) + /(4) + /(5) -/(4, 5, ^^;) + 2/(w;). 
Next notice that /(I, 2, 3, 4, 5, w) - /(2,5,w) < /(3, 4|2, 5, w;) = /(4|2,5,w) 
< /(4|5,w) = /(4,5,w) - f{5,w). Thus /(I, 2, 3, 4, 5, w;) < /(2,5,w;) + 
/(4, 5, w) - /(5, w) < /(4, 5, w) + m + /(5, w) - /(5, w) = /(4, 5, w) + /(2). 
Adding the two inequalities we get: 2/(1, 2, 3, 4, 5, w) < /(I) + /(2) + /(3) + 
/(4) + /(5) + 2f{w). So far the calculation was essentially identical to the 
calculation for the graph entropy with one additional node assigned a variable 
Xyj & W . Now the special information equation for calculating the index code 
allows us to deduce that /(1, 2, 3, 4, 5, w) = /(1, 2, 3, 4, 5) = /(I) + /(2) + 
/(3) + /(4) + /(5) = 5 and to conclude that 2 x 5 < 5 + 2f{w) i.e. that 
2.5 < f{w) and that is{C^) > 2.5. 

It is not hard to adopt the guessing strategy for the guessing game on C5 
and for s e {4, 9, 16, 25, . . .} broadcast the sum (modulo a/s) of each of the 
five L-R pairs. This shows that i(C5,2) < 2.5 X 

Proposition 5A : 

g{G, s) = E{G, s) > EP^^^'^iG, s) > gP^^^'^G, s) 
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Proof: We already showed g{G, s) = E{G, s) (Theorem 1) and Ep^^^^^{G, s) > 
gPubiic^^Q^ g-^ (Proposition 4). It suffices to show that g{G,s) > Ep^''^^%G , s) . 
Before we show this let us observe that g{G, s) > gP^^'-^^{G, s) (this was also 
noticed in [20], [19]). This is because one message Wq of the m possible pubhc 
messages must occur with probability at least ^. If the players guess using 
the same strategies as in the pubhc game, but with each player pretend- 
ing that the public message is wq, they all win with probability > ^ and 
g{G,s) > log,(m). 

We want to show that giG,s) > Ep^^^'%G,s) = n - ientroiG,s). As- 
sume that {pi,P2,P3, ■ ■ ■ ,Pu) is a probability vector and < g < 1 and 
SjPjlog^(^) < \ogg{^). Then there exists j such that pj > q (just view the 
left hand side as a convex linear combination of the logg{pj). Thus for some 
j we must have log^(^) < log^(^)). If the players assume that the public 
message is wq, where this message happens to be broadcasted with probabil- 
ity at least as high as q where q is chosen such that g{G,s) = n — log^(^). 
This shows that g{G, s) > EP''''^'%G, s). X 

The private entropy E{G, s) and the public guessing number 
^pmWicj^^^ s) seem to be identical for many small graphs. This is, however, 
not always the case. 

Proposition 6 : There exists a graph G with distinct private entropy and 
public guessing number (over a fixed alphabet). More specifically 
giC,,2) = EiC,,2) > gP->^^'%C,,2) 

Proof: We already calculated E{C5,2) = log2(5) = 2.32.... Since 
£'P"^'*'^(C5, 2) < 5fP"^'*'^(C5, 2) and since the public guessing number is on the 
form 5 — log2(?72,) the only two serious possibilities for the public guessing 
number are that gP^^^'^{G^,2) = 5 - log2(7) = 2.19... or ^^"'''^^(Cs, 2) = 2. 
Thus ^P"''^*^(C5,2) < 5-log2(7) < log2(5) = E{G^,2) i.e. the entropy (guess- 
ing number) is distinct from the public guessing number. X 
Our computer calculations showed that many small graphs have integer en- 
tropy. This can (partly) be explained by combining the fact that the private 
and public entropy tend to measure very similar notions with the following 
proposition: 

Proposition 7 : // the private entropy E{G, s) is identical to the public 
guessing number (/^"^'^'^(G', s) and s is a prime, then E{G, s) is an in- 
teger. 
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Proof: Let A be an alphabet with s letters. The private entropy is of the 
form log5(mi) while the public guessing number is of the form n — log^ (777,2) 
for 7771,777.2 G {0,1,2,...}. If the private entropy is identical to the public 
guessing number, we have 77 = logg(7ni) + \ogg{m2) = log^(mi7n2) i.e. = 
777 i 7772. If s is a prime number, this is only possible if mi as well as 7772 is a 
power of s i.e. if log^(mi) and log^(7772) are both integers. X 

8 Graph Entropy and its link to Network Cod- 
ing 

8.1 Historical background 

Network Coding is based on the idea that for many network topologies mes- 
sages can be transmitted more efficiently using coding functions instead of 
being transmitted in discrete packets (like in traditional routing). This basic 
observation is often credited in the Network Coding literature to the highly 
influential paper [2S] about Distributed Source Coding for satellite commu- 
nications. It should be pointed out that - if we disregard issues about poten- 
tial applications - the idea was known to the Circuit Complexity community 
at least 25 years before. Curiously in many of Valiant papers including 
[22] where Valiant anticipated the definition of the minimal index code of a 
graph. Valiant considered problems that mathematically clearly presuppose 
the basic observation in Network Coding. 

The main result in [26] (see also [13]) was to show that Network Coding 
for each specific unicast problem - provided the underlying alphabet is suffi- 
ciently large - never leads to "unnecessary" information bottle necks. More 
specifically, they showed that as long as the min-cut capacity between the sin- 
gle source node and each specific target node allows transmission (i.e. there 
are no obvious obstructions), simultaneous transmission from the source node 
to all target nodes is always possible. 

In our view the real significance of [26] was to recognize and propose 
Network Coding as an alternative approach to data management in commu- 
nication networks. Later the notion of Network Coding has been broadened 
and now include, ideas in wireless communication, peer-to-peer files sharing 
(e.g. as in Avalanche by Microsoft). For a general introduction to Network 
coding see for example [25] or [TO] . 
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8.2 Solvability and graph entropy 



A multiple unicast network problem is an acyclic graph with k input nodes 
and k output nodes. Each input is demanded exactly at one output. Each 
node (except for the input nodes) is assigned a coding function and functions 
compose in the obvious manner. 

The acychc independence number of a graph G = (V, E) is defined as 
the size of the maximal induced acyclic subgraph of G [2]. In general if 
i? C V we can obtain a network by duplicating 5, such that each node 
in B is copied into two nodes, one being an input node and one being an 
output node. We say i? is a split if the resulting network N is acyclic (i.e. 
if the induced graph on V" \ i? is acyclic). The number of nodes in B is the 
size of the split. A minimal split is a split of minimal size and the minimal 
split of a graph is the size a minimal split. The minimal split of an acyclic 
graph is since the empty set can serve as a split set. 

12 3, ^ 45 






(i) (ii) (iii) 

As an example consider the communication Network (i) where the inputs 
in vertices 1,2 and 3 at the to are required at vertices 1,2 and 3 at the bottom. 
The network is unsolvable since all messages have to pass through nodes 4 
and 5 that create an information bottleneck. Consider the graph (ii) that 
appeared by identifying each input node with its corresponding output node. 
If the graph (ii) is split with split set {4,5} we get network (iii). This 
network is clearly solvable which shows that graph (ii) has entropy 2. 

We can show (see [2T] and [13] for a related result): 

Theorem 8 : Let G he a directed graph, and let Nq he a communication 
network (circuit) that occurs hy splitting G with a split of size k. Fix 



^"^the task (G, r) of deciding if the input graph G has independence number > r is 
NP-complete [2] 
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an alphabet A and assume that guessing functions gi, g2, . . . , gn are as- 
signed to each node in G. Further, assume that the very same functions 
(with the arguments in a specific order that depends on the split) are 
assigned to the corresponding computational nodes in Nq. Then the 
following are equivalent: 

(i) The probability that by using the guessing strategy determined by 
91, 92, ■ ■ ■ , gn o-ll players guess correctly their own assigned values is (at 
least) ^n-k 

(a) The coding functions gi o ti, g2 o T2, . . . , gn ° Tn define a solution 
to the Network Nq where ri, . . . , r„ are suitable permutations of the 

arguments determined by the split. 

If the split is not minimal (i) and (ii) are never satisfied and the theorem 
becomes vacuous. 

Proof: Assume that the players use the functions gi, g2, . . . , gn as guess- 
ing functions on the graph G = {V,E). Let B be the spht. The induced 
graph on V \ i? is acychc. Without loss of generality we can assume that 
the nodes 1,2, ... ,k belong to B while the nodes k + 1, k + 2, . . . ,n be- 
longs to V \ B. Assume that the very same functions g\,g2, . . . ,gn are 
assigned as coding functions in the corresponding network Nq- Let x = 
) G A" be a random assignment of values to the nodes in G. 
Let {zi{x) , Z2{x) , . . . , Zn{x)) G A" denote the "guess" by the players deter- 
mined by the guessing functions gi, g2, . . . , gn- 
Condition (i) can then be stated as: 

p{zi = Xi, Z2 ^ X2, . . . , Zn ^ Xn) = (^)""*' (13) 

and condition (ii) can be stated as: 

p{zi — Xi, Z2 — X2, . . . , Zk — Xk\Zk^i — Xfe+i, Zk^2 — ■ ■ ■ , — Xn) — 1 

(14) 

In general P{U\V) = and thus 

p{Zi = Xi, Z2 = X2, . . . , Zk = Xk\Zk^i = Xk+1, Zk+2 = ■ ■ ■ ■, = X^) = 

p{zi = Xi, Z2^X2,...,Zn^ Xn) ^^^^ 
P(^fe+1 = ^k+\-i Zk+2 — Xk+2, ■ ■ ■ , Zn — Xn) 
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In general, for any choice of guessing functions (coding functions) we have 



p{Zk+l = Xk+l, Zk+2 = Xk+2, . . . , Zn = Xn) = {-T ^ (16) 

since the induced graph of G restricted to the nodes fc + l,fc + 2,...,?T,is 
acychc so the involved probabilities are independent. Substituting (16) into 
(15) we get that 

(-)" V(^l = Xl^Z2 = X2, . . . , Zk = Xk\Zk+l = Xfc+i, 2:^+2 = Xk+2, ■ ■ ■ , = Xn) 
S 

= p{Zi = Xi,Z2 = X2, . . . ,Zn = Xn) (17) 

i.e. that (13) holds if and only if (14) holds. X 
To illustrate the theorem, consider the Networks (b)-(d) and the assign- 
ments of guessing functions gi, g2, . . . , for the graph a. Each Networks 
(b)-(d) appears by "splitting" the graph a in different ways. 



Consider as an example Network b The network in b was introduced in 



[HI [7] where the authors showed that the network has only linear solutions over 
fields of characteristic 2 (i.e when the alphabet contains 2^ elements for some 
r G {1, 2, 3, . . .}). Our concern is to illustrate Theorem 8. Let us recall that 
solving network b consists of finding coding functions gi, 92, ■ ■ ■ , 97 such that 
for any choice of input messages Xi,X5,X2 G A, (for nodes 1,5 and 2 at the 
top) these messages can be reproduced at their corresponding output nodes 
(nodes 1,5 and 2 at the bottom). Since each vertex has two predecessors, 
each coding function is a map 9j : A. The coding functions compose in 

the obvious way and they solve the network if for any input (xi,X5,X2) the 
output (viewing the network as a circuit) is {x2,x^,Xi) (reading from left to 
right). 



^^we prefer to represent multiple unicast problems in Network coding by using the line- 
graph. This way coding functions are attached to the nodes rather to the edges 
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(a) (b) 



2 6 3 1 3 5 




(c) (d) 

If we in Network b identify the three sources at the top (nodes 1, 2 and 5) 
with the corresponding output nodes at the bottom (nodes 1, 2 and 5), we get 
the graph a. The functions gi,g2, ■ ■ ■ ,97 ai'c assigned to the corresponding 
nodes in Network b. The spilt {1,2,5} leads to network b while the split 
{2,3,6} leads to network c. Other splits e.g. {3,4,7} lead to a network 
that is isomorphic to network a and network b . The split {1,3,5} leads 
to network d and the split {2,4,6} would lead to Network isomorphic to 
network d. We assigned guessing functions to the nodes in the graph a. 
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Notice that each spht leads to different assignments of coding functions in 
the resulting network. Please keep in mind that the arguments of coding 
functions has to be defined appropriately. Let t{u,v) = {v,u) and define in 
general h := h o r. Then if we define the coding functions such that they 
take the left incoming value as first argument, and right incoming as second 
argument, we get a network coding solution for Network b by assigning the 
coding functions gi,g2, 93, 94., 95, 96 and gj as indicated. 

One direct consequence of theorem 8 is that the functions gi,g2, ■ ■ ■ ,97, 
provide a solution for one of the networks (6), (c) or (d) if and only if they 
provide a solution for each of the networks ({b),{c) and (d)). Combining 
Theorem 1 that the entropy is identical to the guessing number, as well as 
Proposition 3, i.e. that the public entropy of a graph G = {V, E) is identical 
to |\^| minus the minimal index code of G (since the minimal index code is 
the very same notion as the information defect [201 IT9|)- Theorem 4 in [19] 
can be rephrased as follows: 

Corollary 9 : A multiple unicast network N with k input and k output 
nodes, is a solvable Network over an alphabet A if and only if the graph 
G]\r has (private/public) entropy k over A 

Corollary 9 is not correct if we remove reference to a specific alphabet A 
and just consider the general Entropy. This follows essentially by considering 
the unsolvable network N with coding capacity 1 constructed in [H]. 

8.3 Links to Network coding capacity 

In [8] the authors define the coding capacity of a communication network. 
Here we are only interested in the special case of coding capacity 1 (more 
specifically coding capacity (1, 1)). Let be a communication network i.e. 
an acyclic graph with a set I of k input nodes and k output nodes. We say 
that N has S-coding capacity (1, 1) (ZY-coding capacity (1, 1)) if there exists 
a S-entropy-like function / (ZY-entropy-like function /) such that 

(*) Conditions N1,N2 and N3 from [8] for (1, 1) coding capacity: 
Nl: /(/) = k, 

N2: f{j) < 1 for each node j 

N3: f{j\ii, 12, . ■ . ,id) = for each node j where Zi, 12, ■ . . ,id are the prede- 
cessors of j. 
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The coding capacity conditions N1,N2 and N3 (for capacity (1, 1) net- 
works) differ of course in general from tfie definition of grapli entropy. The 
coding capacity is defined for acychc networks, with special input nodes, 
rather than for graphs. Condition Nl for calculating the coding capacity as- 
sumes that the certain nodes are assigned independent variables, while there 
is no such assumption in the definition of graph entropy. On the other hand, 
the graph entropy assumes that a certain entropy is being optimized, while 
there is no such assumption in the definition of the coding capacity. How- 
ever, the two notions Graph entropy and Coding capacity are in some sense 
identical in the special case where the maximal value of /(I, 2, . . . , n) = /(/) 
(in which case condition Nl holds). 

In other words, if we disregard condition Nl and just try to maximize 
/(I, 2, . . . , n) = /(/) like we need to calculate the graph entropy, then since 
the network is assumed to have S-coding capacity (1, 1) (ZY-coding capacity 
(1, 1)) we conclude that /(/) = k which ensures that condition Nl is auto- 
matically satisfied. Conversely, assume that Gn has S-entropy k (ZY-entropy 
k). But then since /(J) = /(I, 2, . . . ,k) = /(I, 2, . . . , n) = k, we conclude 
that the conditions N1,N2 and N3 are satisfied. Hence: 

Proposition 10 : A network N with k input-output pairs, has shannon 
coding capacity (ZY-coding capacity) (1, 1) if and only if the graph Gn 
that occurs by identifying each input node with its corresponding output 
node has S-entropy (ZY-entropy) k (or > k). 

Theorem 11 : There exists a graph G with Ezy{G) < Es{G). Thus it 
has a general entropy E{G) that cannot he calculated using Shannon's 
information inequalities. 

Proof: From [8] we know that the Vamos communication network N' modi- 
fied to the multiple-unicast-situation has S-capacity 1, but has strictly smaller 
ZY-capacity (at most 12/13). The network can be represented as a circuit 
(by passing to the line graph). This does not change anything since the in- 
volved coding functions are the same and the resulting circuit N still has 
S-capacity 1 and strictly less ZY-capacity (that is at most 12/13). We then 
identify each source node with its corresponding target node and obtain a 
graph Gn- It follows from Theorem 10, that since the S-capacity of N is 
(1, 1) - the S-entropy of G^ is k (where k is the number of input nodes in 
A^). The ZY-capacity of is not (1, 1) - so again according to Theorem 10 
the ZY-entropy of Gat is strictly less than k 4> 
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We do not attempt to calculate the specific difference between G"s S- 
entropy and ZY-entropy, and this is left as an open problem. 

9 Final remarks and acknowledgments 

Proposition 10 and Theorem 11 as well as many concepts introduced in the 
paper depends strongly on the work by Dougherty, Freiling and Zeger ( [6|, [3 
E] ) as well as the earlier work by Zhang and Young ( [28] ) . The use of entropy 
functions for multiple-unicast networks was introduced in [6]. 
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